Elastic Processing of Analytical Query Workloads on IaaS Clouds
نویسندگان
چکیده
Many modern applications require the evaluation of analytical queries on large amounts of data. Such queries entail joins and heavy aggregations that often include user-defined functions (UDF s). The most efficient way to process these specific type of queries is using tree execution plans. In this work, we develop an engine for analytical query processing and a suite of specialized techniques that collectively take advantage of the tree form of such plans. The engine executes these tree plans in an elastic IaaS cloud infrastructure and dynamically adapts by allocating and releasing pertinent resources based on the query workload monitored over a sliding time window. The engine offers its services for a fee according to service-level agreements (SLAs) associated with the incoming queries; its management of cloud resources aims at maximizing the profit after removing the costs of using these resources. We have fully implemented our algorithms in the Exareme dataflow processing system. We present an extensive evaluation that demonstrates that our approach is very efficient (exhibiting fast response times), elastic (successfully adjusting the cloud resources it uses as the engine continually adapts to query workload changes), and profitable (approximating very well the maximum difference between SLA-based income and cloud-based expenses).
منابع مشابه
VM-MAD: A Cloud/Cluster Software for Service-Oriented Academic Environments
The availability of powerful computing hardware in IaaS clouds makes cloud computing attractive also for computational workloads that were up to now almost exclusively run on HPC clusters. In this paper we present the VM-MAD Orchestrator software: an open source framework for cloudbursting Linux-based HPC clusters into IaaS clouds but also computational grids. The Orchestrator is completely mod...
متن کاملCost Minimization in Multiple IaaS Clouds: A Double Auction Approach
Abstract—IaaS clouds invest substantial capital in operating their data centers. Reducing the cost of resource provisioning, is their forever pursuing goal. Computing resource trading among multiple IaaS clouds provide a potential for IaaS clouds to utilize cheaper resources to fulfill their jobs, by exploiting the diversities of different clouds’ workloads and operational costs. In this paper,...
متن کاملCost-effective Resource Configurations for Executing Data-intensive Workloads in Public Clouds
........................................................................................................................................... ii Co-Authorship ................................................................................................................................ iv Dedications ...................................................................................................
متن کاملPerformance Evaluation of Cloud Infrastructure using Complex Workloads
Infrastructure as a Service (IaaS) is a delivery model of cloud computing, which provides the ability to users to acquire and release resources according to their demand and pay according to their usage. Resources are provisioned from the cloud as Virtual Machines (VMs), many of which can be deployed on a single computing node, realizing a multi-tenancy model. While virtualization and multitena...
متن کاملA Method for Measuring Energy Consumption in IaaS Cloud
The ability to measure the energy consumed by cloud infrastructure is a crucial step towards the development of energy efficiency policies in the cloud infrastructure. There are hardware-based and software-based methods of measuring energy usage in cloud infrastructure. However, most hardware-based energy measurement methods measure the energy consumed system-wide - including the energy lost in...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1501.01070 شماره
صفحات -
تاریخ انتشار 2015